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ASYNCHRONOUS COLLABORATION VIA AUDIO/VIDEO ANNOTATION 

Joon Maeng 
BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to the field of electronic communication and 
more specifically to communication of annotated audiovisual information to and from 
remote users. 

Description of the Related Art 

A significant development in computer networking is the Internet, which is a 
sophisticated worldwide network of computer systems. There are several different 
types of information that may be communicated via the Internet or other computer 
network, including video mail, electronic mail, and web pages, which may be made 
up of various types of content and presentation formats. 

One of the benefits of electronic communication, such as electronic mail (e- 
mail) is that a user may attach files to a message. These files may include documents, 
web pages, and the like. One drawback to the provision of such files is that the files 
must be sent as an attachment to an e-mail, and cannot be edited directly, that is, text 
notes or other information relating to the file must be sent as an attachment. 

A benefit of a computer network is that a plurality of remote entities or users 
may communicate quickly and efficiently. Further, a group of such users may 
collaborate on a project remotely. For example, one user may create an electronic 
textual file by use of any known word processing application program, and then e- 
mail the file as an attachment to the message. Then, the other users can provide 
comments, and send a revised file back to the original sender. However, this process 
leads to inefficiencies. 
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SUMMARY OF THE INVENTION 

In accordance with the present invention, a system is provided to allow remote 
entities to asynchronously collaborate with regard to a desired file. More particularly, 
the present invention permits a user to create an audiovisual message and deliver it as 
5 a combined file with the desired file. As used herein, the term "v-mail," "video mail" 
or "audiovisual message" means an electronic file containing at least audio and video 
information and may also include textual and/or graphical information. The message 
may then be delivered directly to recipients via a standard e-mail or v-mail program, 
or the message may be provided to a central location, which provides for 
10 management, control and delivery of the message to the selected recipients. 

In one embodiment, the present invention includes method of asynchronous 
communication by a plurality of users via a computer network, including the steps of 
selecting a file and capturing it as an image file; creating an audiovisual message 
relating to the file; appending the image file and the audiovisual message to achieve a 
15 combined message; and electronically delivering it to at least one of the users. 

Further, the present invention includes a method of transmitting a 
concatenated audiovisual file via a computer network, including the steps of locating 
a first audiovisual file having a first identification code and a plurality of frames; 
locating a second audiovisual file having the first identification code, a first frame 
20 reference number, and a first reply portion; transmitting at least a portion of the first 
audiovisual file via the network until the first frame reference number is attained; and 
transmitting the first reply portion after the first portion. 

Additionally, an embodiment of the present invention includes a system for 
the creation and delivery of an audiovisual message, including remote workstations 

25 adapted to permit a client to capture selected content, annotate the content, create an 
audiovisual message, combine the content, annotation, and audiovisual message as a 
linked message, and deliver it to at least one remote recipient; a central server 
connected to the remote workstations, and being adapted to receive the linked 
message and to receive at least one reply message; and a central storage location 

30 having a relational index to correlate reply messages to the linked messages. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention may be better understood, and its numerous objects, 
features and advantages made apparent to those skilled in the art by referencing the 
accompanying drawings. The use of the same reference number throughout the 
several figures designates a like or similar element. 

FIG. 1 is a block diagram of an example of a computerized information 
network in accordance with the present invention. 

FIG. 2 is a representation of an example graphical user interface that may be 
used in one embodiment of the present invention. 

FIG. 3 is a block diagram of an example screen display of a message in 
accordance with the present invention. 

FIG. 4 is a block diagram of an example network that may be used in 
accordance with the present invention. 

FIG. 5 A is a block diagram of an example embodiment of a file format of a 
video message in accordance with the present invention. 

FIG. 5B is a block diagram of an example embodiment of a file format of a 
reply to a video message in accordance with the present invention. 

FIG. 5C is a block diagram of an example embodiment of a file format of a 
video message in accordance with the present invention, showing the location of 
preselected frames. 

DETAILED DESCRIPTION 

While the Internet is used herein as an example of how the present invention is 
utilized, it is important to recognize that the present invention is also applicable to 
other electronic communication applications, such as video teleconferencing, 
computerized information networks, and the like. A brief overview of concepts 
pertaining to the Internet, the world-wide web, and web servers is presented to 
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introduce terminology used throughout the description and claims of the present 
invention. 

An example of a typical connection of components in an electronic network 
100, such as the Internet, is shown in FIG. 1. A user that wishes to access information 
5 via the electronic network 1 00 may communicate through a computer network 110, 
such as the Internet. The user typically uses a client workstation 112 that executes 
application programs, such as a browser and an electronic mail program, for example. 
Of course, a plurality of workstations 112 may be interconnected so that plural users 
may communicate. Workstations 1 12 may be connected via any one of numerous 
10 communications links, such as a dial-up wired connection with a modem, a direct link 
such as a Tl, ISDN, or cable line, a wireless connection through a cellular or satellite 
network, or a local data transport system such as Ethernet or token ring over a local 
area network. 

As shown in FIG. 1, a user may use client workstation 1 12, which may be a 

15 personal computer, Internet appliance, personal digital assistant, or the like, to access 
remote content, such as a web site 114. In an example embodiment, the web site 114 
may contain content including audio and visual information. Further, the web site 
1 14 may be arranged as a group of web pages 115, each relating to a particular subject 
or the like. It is to be understood that the web site and web page may be resident on a 

20 remote server computer, as is well known in the art. A web page is primarily visual 
data that is intended to be displayed on a display device, such as the monitor of user 
workstation 112. When a web page request is received, a document, generally written 
in a markup language such as hypertext markup language (HTML) is transmitted 
across the communication link to the requesting browser. The browser interprets the 

25 markup language and outputs the web page to the monitor of user workstation 112. 
This web page displayed on the user's display may contain text, graphics, and links 
(which are addresses of other web pages). The user can go to these other web pages 
by clicking on the links using a mouse or other pointing device. This entire system of 
web pages with links to other web pages on other servers across the world is known as 

30 the "World Wide Web". 
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A user may also access information located at a remote location of a service 
provider 1 1 6 via the network 110. For example, the user may desire to access a mail 
server 118 located at the service provider 1 16, in order to obtain access to electronic 
mail, video mail, or the like. 

5 The preceding discussion relates to well known features and components used 

in typical computing environments, and more particularly for use in a computer 
network. One benefit of such a computer network is that a plurality of remote entities 
or users may communicate quickly and efficiently. Further, a group of such users 
may collaborate on a project remotely. As discussed above, a user may create a text 
1 0 file and e-mail the file as an attachment to the message. Then, other users can provide 
comments, and send a revised file back to the original sender. However, this process 
leads to inefficiencies. Accordingly, the system disclosed herein allows remote users 
to collaborate asynchronously, by allowing users to send and receive audiovisual 
messages which may themselves include comments regarding an associated file. 

15 In one example of the present invention, a user may desire to provide 

annotation to a file which it seeks to provide to one or more recipients. For example, 
a user may wish to send a web page to co-workers. Further, the user may wish to 
send the web page with annotation, such as editing, markup, or highlighting of the 
web page, and also include an audiovisual message, or a combination thereof. Upon 

20 receipt of such a combined message, one or more of the co-workers may similarly 

annotate the message, create an audiovisual reply and forward the reply to some or all 
of the original recipients. In this way, a user may thus collaborate asynchronously 
with the recipient(s) of the message. 

In a more particular embodiment, a user may create a combined message to 
25 send to one or more users via a computer network. Specifically, the user may capture 
content of a particular file (such as a web page, document, or the like) into an image 
file, then add annotation to the image file (such as text, highlighting, editing, 
commenting, or the like), then add an audiovisual mail message to the image file, and 
send the combined message to the selected user or users. In addition, the recipient(s) 
30 may also add annotation and provide an audiovisual message in response. In certain 
embodiments, the recipient may provide a reply with an attached address pointer, 
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indicating a preselected point in the original message at which the reply should be 
played, so that the amount of data included in the reply may be reduced. 

To create a combined message, the user may access a graphical user interface 
(GUI) which provides access to the annotation and audiovisual features of the present 
5 invention, and which may be a WINDOWS-based software module. Further, such a 
GUI may be implemented as an ActiveX control embedded within a web page or the 
like. Alternately, a Java Media Framework-based applet, a standalone client 
application, or a NETSCAPE plugin may be used to access the software of the 
system. In example embodiments, a user may access the GUI at a client workstation 
10. or the like via a screen icon, hot key, pull-down menu, or in any other manner. 

FIG. 2 is an example embodiment of a GUI 10 for use in connection with the 
creation of video messages. As shown in FIG. 2, the GUI may contain a "To" field 
12, so that the user can enter the address of intended recipient(s). Addresses may be 
obtained via a preexisting address book, such as available in MICROSOFT 

15 OUTLOOK. Further, it is to be understood that other fields not shown in FIG. 2, such 
as "CC" and "BCC" may be present in certain embodiments. Additionally, a 
"subject" field and a field for inserting text may also be present in certain 
embodiments. In certain embodiments, voice recognition may be used so that the user 
may simply speak the names of the intended recipients, and the recognition software 

20 will select the recipients from a list. 

Also shown in FIG. 2, the GUI 10 may contain a voice level indicator 14, a 
microphone volume selector 16, a mail length indicator 18, video window 22, and a 
plurality of stateful buttons 20. The microphone volume selector 16 may be 
controlled via a cursor or the like to control the sensitivity of the microphone input. 

25 However, it is to be understood that in other embodiments, the sensitivity may be 
automatically controlled by an automatic gain controller (AGC), and that the voice 
level indicator 14 and the microphone volume selector 16 may be hidden. The video 
window 22 may actively display the view of a video camera or other capture device 
connected to the computer, so that the user may see and adjust his or her image. It is 

30 to be understood that in example embodiments, a GUI may contain more or fewer 
indicators, fields, and the like. 
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In an example embodiment, a user may utilize capture devices connected to 
his or her workstation to record the audiovisual information and transform it into a 
digital format. These capture devices may include a camera device having a 
composite video output, such as available from Sony Corporation, or a USB camera, 
5 an example of which is available from Intel Corp. (Santa Clara, CA). Further, a 

microphone connected to the computer may be used to capture audio. In an example 
embodiment, a user's default capture devices as specified in the WINDOWS 
Multimedia control panel may be used. In an example embodiment, the capture 
bandwidth may be set at 56 KB/sec. using H.263 to encode video and GSM (for 
10 example, G.728 or G.723.1 protocols) to encode audio. However, it is to be 
understood that other bandwidths and protocols may be used, such as higher 
128KB/sec. or 384KB/sec. bandwidths, and MPEG encoding of video. 

Additionally, it is to be understood that each user's workstation may have 
appropriate audiovisual playback equipment, in addition to capture devices. This 
1 5 equipment may include a video monitor, audio speakers, and the like. Further, the 
workstations may have appropriate audio and video encoders and decoders, as is well 
known in the art, and described in U.S. Patent No. 6,014,689, the disclosure of which 
is hereby expressly incorporated by reference. 

Further, in example embodiments, the audio and image data may be 
20 compressed using a variety of distinctly different compression schemes to save 

storage space, as known in the art. The data may also be encrypted to help prevent 
unauthorized access to the data. Well known application programs may be used to 
compress, decompress, encrypt, and decrypt data, as required. 

In operation, the user may perform desired tasks using the stateful buttons 20, 
25 which are graphical representations of functions available, and which may be selected 
by clicking a mouse, key, or other pointing device. In an example embodiment, a first 
state may exist that includes three stateful buttons 20, namely a "Start - A/V Only," a 
"Start Recording," and an "Exit" button. When the user selects the "Start - A/V 
Only" button, recording is begun to create a message without any attachment. 
30 Selection of the "Start" button begins recording, and includes a selected file for 
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attachment to the message. When the user selects either of the start buttons, the state 
changes to a second state. 

In the second state, a "Pause," "Cancel," and "Send" button may appear on the 
GUI. The "Pause" button may be used to pause recording, the "Cancel" button 
5 terminates the recording without saving the message, and the "Send" button sends the 
message to the preselected recipients. When the "Pause" button is selected, a third 
state may be entered in which four new buttons may appear on the GUI. Specifically, 
these buttons may include a "Resume," a "Cancel," an "Exit," and a "Send" button. 
The "Resume" button continues the recording from the position where it was paused, 

10 and control returns to the second state. The "Cancel" button, in any state, returns 

control to the first state. Further, the "Exit" button is used to exit the GUI. It is to be 
understood that in other embodiments, there may be more, fewer, or different 
functions and states available. For example, a preview function may be made 
available to permit a user to preview a recorded message before sending it. Further, a 

1 5 GUI having similar functions may be used to provide a user with various playback 

options, including allowing a user to periodically pause playback in order to create an 
audiovisual reply. 

In example embodiments, when the "Start" button is selected, a cursor on a 
video screen may change into a highlighter or other markup device, so that the user 

20 may highlight or otherwise markup portions of the file to be appended to the message. 
Upon activating the highlighter, the user may use a mouse or other pointing device to 
control movement of the highlighter, with the left mouse button (or the like) being 
used to start/stop highlighting, while the mouse is dragged to highlight the selected 
material. Additionally, upon entry into the GUI, the selected file may be activated to 

25 allow the user to edit or otherwise modify the file using well known tools such as are 
available in a word processing or other application program. 

FIG. 3 is an example embodiment of a video display of a combined message, 
as viewed by a recipient. As shown in FIG. 3, the display 40 includes an image file 
42, an annotation of text information 44 including highlighted portions 45, and a 
30 video inset 46, which may be played by the recipient to view an audiovisual message 
forwarded by the sender. As shown in FIG. 3, the combined message appears as an 
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integrated image; that is, there is no attachment to the message, instead the message is 
a single cohesive display. Although shown having an image file, annotation, and an 
audiovisual message, it is to be understood that in certain embodiments, one or more 
of these fields may not be desired. 

5 It is to be understood that delivery of audiovisual messages may be 

accomplished via standard electronic mail programs and the like. In an example 
embodiment, however, a system may be provided via a computer network having a 
client/server architecture, wherein the created messages are provided from remote 
client sites to a central server for management, storage and delivery. Such a central 
10 server site may be an application service provider ("ASP") or may be internal to a 
given company or entity. 

FIG. 4 is a block diagram of an example client/server system 300. As shown 
in FIG. 4, the system 300 includes a remote client site 310 (of course, a plurality of 
such sites may be present in a given embodiment), which in an example embodiment 

15 may be a workstation such as a personal computer, Internet appliance, personal digital 
assistant ("PDA"), or the like. The remote client site 3 1 0 is interconnected to a 
central server 315 of a provider or the like via a desired computer network, such as 
the Internet. The central server 315 may be a standard server computer operating on a 
WINDOWS, UNIX, or LINUX platform, such as an IBM RS/6000, AS/400, DELL 

20 POWEREDGE, or the like. The central server 3 1 5 may include post office software 
320, which may act as a central interface for the system, and be responsible for the 
managing of messages, and control of access to the same. 

Additionally, as shown in FIG. 4, the system may include a storage center 330 
for the storage of messages 332. In an example embodiment, the storage center 330 

25 may be part of the central server 3 15, or may be an external mass memory storage, 
such as a storage area network (SAN), such as a DELL POWERVAULT, or the like. 
Additionally shown in FIG. 4 is a remote recipient site 340, which may be a 
workstation, such as a personal computer, Internet appliance, PDA, or the like. Of 
course, in example embodiments a plurality of client sites 310 and a plurality of 

30 recipient sites 340 may be present. In sum, the system may be thought of as a 

plurality of Clients 310 and/or 340 connected to a Post Office 320, which in turn is 
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connected to a Storage Center 330. In an example embodiment, the storage center 
330 may make use of a relational database management system using an index system 
to provide safe, efficient control of the messages. 

In an example embodiment, the post office software 320 may provide for 
5 storage of in-transit messages, and be responsible for generating notification messages 
when a new message for a particular recipient arrives. Additionally, the post office 
software 320 may act as a media server from which streaming playback may occur to 
provide a recipient with audiovisual data. Upon receipt of a new video mail message, 
the software 320 may send electronic mail messages to recipients that contain a 
10 clickable Uniform Resource Locator (URL) which provides a hyperlinked address so 
the recipient may access the message in the Storage Center 330. 

Upon receiving an access request from a recipient, the central server 315 
accesses the appropriate message from the storage center 330, and delivers it (via the 
Internet or other computer network). In an example embodiment, the message may be 

1 5 streamed to the recipient(s). Recipients may play back messages using a viewer 

applet, such as the Vviewer applet available from VTEL Corp. (Austin, TX) as part of 
the TURBOCAST suite of products. Use of such an applet obviates the need to 
download and install helper applications or plugins. Alternately, other known 
viewers, such as MICROSOFT MEDIA PLAYER, REAL PLAYER, or an applet 

20 viewer with the message attached may be used. By streaming the video to a recipient, 
there is no need to include large video files within messages. The applet may permit a 
recipient to perform simple play/pause functionality, and the ability to seek backwards 
and forwards within the message. 

As discussed above, a recipient may reply to a message with audiovisual 
25 material, using the same general techniques described above for creating a message. 
As with a standard e-mail reply, which is sent with the original message, the reply 
message may be sent with the original message attached. However, in certain 
embodiments, the recipient may choose to have the reply concatenated onto the 
message, so that when viewed, reply portions will appear at a preselected point in the 
30 message. For example, a message may contain a discussion of three topics. A reply 
to the message may also contain a discussion of each of the topics. Using the present 
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invention, a recipient may develop its reply to correspond and concatenate it to each 
of the topics. 

Shown in FIG. 5 A is a block diagram representation of a video file or record 
format. As shown in FIG. 5A, a video file 50 includes a file identification code 52 
5 and a plurality of bytes 54 containing digital representations of information (i.e., 
video, audio, text, or a combination thereof). 

In operation, video data is viewed by a recipient at 30 frames per second, as is 
standard. When a recipient wishes to respond at a certain point in the streaming video 
message, he or she can access a GUI, as discussed above, and use the stateful buttons 
10 to create a reply. In an example embodiment, playback of the message will be paused 
while the recipient creates a reply. At the start of the reply process, the GUI will store 
a file identification code corresponding to the message being played, and a frame 
reference number, which is the point in the video playback at which the recipient 
wishes to reply. 

1 5 Upon completion of a reply portion, the recipient may continue viewing the 

message, and may use the GUI to again stop the message at a particular position, so 
that another reply portion to some other portion of the message may be created. This 
process may be performed iteratively. At the conclusion of the message, a recipient 
may have a reply message that contains a plurality of reply portions. As discussed 

20 above, each of these reply portions may have a corresponding file identification code 
and a frame reference number associated with it. In this way, the completed reply 
message may be sent without the original message to reduce file size. 

The recipient may then send the completed reply message to a list of 
individuals, including the author of the original message, other recipients, or other 

25 interested individuals. In an example embodiment, the reply message may be 

provided to central server 315, which will perform the management functions. These 
functions may include advising recipients of the reply that it is available for viewing 
(for example, via e-mail with a URL hyperlink). Additionally, the post office 
software 320 will provide the reply message to the Storage Center 330. Using the 

30 relational database architecture of the Storage Center 330, the reply will be stored so 
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that it may be accessed in conjunction with the original message. Of course it is to be 
understood that the reply message may be viewed by its recipients independently of 
the original message, at the conclusion of the message. 

FIG. 5B shows a reply message 60 that contains three independent reply 
portions, Rl, R2, and R3, each of which corresponds to a different portion of the 
original message. As shown in FIG. 5B, the reply message 60 contains a file 
identification code 52 (which will be identical to the identification code of the original 
message to which it is responding), frame reference numbers 62, and reply portions 
64. The frame reference numbers 62 correspond to the point (or "preselected frame") 
in the original message at which a reply was initiated. FIG. 5C shows the preselected 
frames (shown as "Frame X," "Frame Y," and "Frame Z") of the original message 50 
after which the reply portions will be shown. 

In operation, when a recipient desires to see a combined message, i.e., the 
original message with concatenated replies, which may be selected from an 
appropriate button on a viewer GUI, the appropriate files will be accessed via the 
Storage Center 330. For example, as shown in FIG. 5C, the system displays a 
message 70, which includes the original message 50 and any replies having the file 
identification code of the original message. Then, the system will show the original 
message 50, until a preselected frame (e.g., "Frame X"), which corresponds to the 
frame identification where the first reply portion Rl was initiated, is reached. Then, 
Rl of reply message 60 will be played, as it is stored with a frame reference number 
62 corresponding to the preselected frame. At the conclusion of the Rl portion, the 
original message 50 will continue, until another preselected frame (e.g., "Frame Y") 
corresponding to another reply portion (e.g., R2) of reply message 60 is reached. In 
this manner, the system provides for asynchronous collaboration, so that a recipient 
may view an original message and replies thereto as a cohesive and concatenated 
message. Further, it is to be understood that a plurality of replies from a plurality of 
users may be concatenated in accordance with the above. 

Other Embodiments 

While the invention has been described with respect to the embodiments and 
variations set forth above, these embodiments and variations are illustrative and the 
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invention is not to be considered limited in scope to these embodiments and 
variations. For example, the invention may be provided via software encompassing 
any computer readable medium, such as CD-ROM, diskette, ZIP disk, tapes, ROM, 
RAM, hard drive and the like. Accordingly, various other embodiments and 
modifications and improvements not described herein may be within the spirit and 
scope of the present invention, as defined by the following claims. 
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WHAT IS CLAIMED IS: 

1 LA method of asynchronous communication by a plurality of users via a 

2 network of interconnected computers, comprising: 

3 selecting, at a client site, a file and capturing said file as an image file; 

4 creating, at said client site, an audiovisual message, said audiovisual message 

5 relating to said file, said creating comprising capturing digital 

6 information; 

7 appending said image file and said audiovisual message to achieve a combined 

8 message; and 

9 electronically delivering said combined message to at least one of said 
1 0 plurality of users . 

1 2. The method of claim 1 , wherein said electronically delivering 

2 comprises streaming said digital information via said network. 

1 3. The method of claim 1, further comprising creating a reply record, said 

2 reply record having audiovisual material, said reply record being in response to said 

3 combined message. 

1 4. The method of claim 3, wherein said reply record is created by one of 

2 said plurality of users, and further wherein said one of said plurality of users annotates 

3 said image file. 

1 5. The method of claim 1 , further comprising annotating at least a portion 

2 of said image file using a cursor-based highlighter. 

1 6. The method of claim 1, wherein said creating comprises accessing a 

2 user interface, said user interface having stateful buttons. 

1 7. The method of claim 1, wherein said selecting further comprises 

2 activating said image file for editing. 
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1 8. The method of claim 1, wherein said electronically delivering 

2 comprises transmitting said combined message to a central server for storage and 

3 further transmission. 

1 9. A method of creating and delivering an electronic audiovisual message 

2 via a network of computing means, comprising: 

3 capturing at least a portion of an electronic document as an image file; 

4 annotating said image file; 

5 creating said electronic audiovisual message, said electronic audiovisual 

6 message regarding said electronic document; 

7 combining said annotated image file and said electronic audiovisual message 

8 into a combined message; and 

9 delivering said combined message to at least one desired recipient via said 
1 0 network. 

1 10. The method of claim 9, further comprising replying to said combined 

2 message via said network, wherein said replying comprises creating an audiovisual 

3 reply designated for insertion at a predetermined point in said combined message. 

1 11. The method of claim 1 0, wherein said replying comprises delivering 

2 said audiovisual reply with an address pointer indicative of said predetermined point. 

1 12. The method of claim 9, wherein said delivering comprises storing said 

2 combined message at a remote location and sending an electronic notification having 

3 a uniform resource locator corresponding to said location. 

1 13. The method of claim 9, wherein said delivering comprises streaming 

2 said combined message to said at least one desired recipient via the Internet. 

1 14. A method of transmitting a concatenated audiovisual file via a 

2 computer network, comprising: 



641299 vl 



15 



Attorney Docket No.: M-8308 US 



3 locating a first audiovisual file stored in a central location, said first 

4 audiovisual file having a first identification code and a plurality of 

5 frames, said frames comprising digital information; 

6 locating a second audiovisual file stored in said central location, said second 

7 audiovisual file having said first identification code, a first frame 

8 reference number, and a first reply portion, said first frame reference 

9 number corresponding to one of said plurality of frames of said first 

10 audiovisual file; 

1 1 transmitting at least a portion of said first audiovisual file via said computer 

12 network, said at least a portion extending until said first frame 

13 reference number is attained; and 

14 transmitting said first reply portion after said at least a portion of said first 

15 audiovisual file via said computer network, thereby transmitting said 

16 concatenated audiovisual file. 

1 15. The method of claim 14, further comprising transmitting a second 

2 portion of said first audiovisual file upon completion of said first reply portion, said 

3 second portion terminating at a second frame reference number. 

1 16. The method of claim 15, further comprising transmitting a second 

2 reply portion upon completion of said second portion. 

1 17. The method of claim 14, wherein said first audiovisual file comprises a 

2 comment relating to a source file on which a group of users are collaborating, and 

3 further wherein said second audiovisual file comprises a reply to said comment. 

1 18. The method of claim 14, further comprising a plurality of audiovisual 

2 files, each of said plurality of audiovisual files having said first identification code 

3 and including at least one reply portion. 

1 19. The method of claim 14, wherein said transmitting comprises 

2 providing streaming video via said computer network. 
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1 20. The method of claim 14, wherein said transmitting occurs via the 

2 Internet. 

1 2 1 . A system for the creation and delivery of an audiovisual message, 

2 comprising: 

3 a remote client workstation adapted to permit a client to capture selected 

4 content, annotate said selected content, create said audiovisual 

5 message, combine said selected content, said annotation, and said 

6 audiovisual message as a linked message, and deliver said linked 

7 message to at least one remote recipient; 

8 at least one remote recipient workstation adapted to permit said at least one 

9 remote recipient to view and reply to said linked message, said at least 

I o one remote recipient workstation comprising at least one capture 

I I device; 

12 a central server interconnected to said remote client workstation and said at 

13 least one remote recipient workstation, said central server adapted to 

14 receive said linked message from said remote client workstation and to 

15 receive at least one reply message from said at least one remote 

16 recipient workstation; and 

17 a central storage location connected to said central server, said central storage 

1 8 location having a relational index, said relational index to correlate 

19 said at least one reply message to said linked message. 

1 22. The method of claim 21 , wherein said at least one remote recipient 

2 workstation is adapted to permit said at least one remote recipient to create an 

3 audiovisual reply having a frame reference number corresponding to a preselected 

4 portion of said linked message. 



1 23 . A machine-readable storage medium comprising a set of instructions 

2 executable by a computer system to implement a method, the method comprising: 

3 providing a graphical user interface to allow a user to select desired content as 

4 an image file; 
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5 permitting said user to annotate said image file; 

6 capturing audiovisual material responsive to said image file as a digital file; 

7 pairing said digital file and said annotated image file as a combined file; and 

8 delivering said combined file to at least one desired location via said computer 

9 system. 

1 24. The computer program product of claim 23, wherein the machine- 

2 readable storage medium includes any of magnetic storage medium, including disk 

3 and tape storage medium; optical storage medium, including compact disk memory 

4 and digital video disk storage medium; nonvolatile memory storage memory; volatile 

5 storage medium; and modulated, electronic signals. 
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ASYNCHRONOUS COLLABORATION VIA AUDIO/VIDEO ANNOTATION 

Joon Maeng 

ABSTRACT OF THE DISCLOSURE 

A system and method is disclosed for asynchronous collaboration between a 
plurality of remote users. More specifically, a system based on a client/server 
architecture is disclosed, in which a remote user may create a combined message 
having an audiovisual message and an image file on which collaboration is to occur. 
The remote user may then transmit the message and file to a central location, where it 
is stored and then accessed by other remote users for viewing, replying, and editing. 
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